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(54) XML based report generator 

(57) An invention is provided for creating a test sum- 
mary report. A test application is executed on a platform, 
where the test application is executed using a status util- 
ity having functions that generates XML code. The test 
results are generated in a XML enabled format using the 



status utility, and are output to a test execution log file. 
The test execution log file is processed to generate a 
well-formed XML test reports file, which is then logically 
arranged to create a logically arranged XML test reports 
file. The logically arranged XML test reports file is con- 
verted into a HTML test summary report. 
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Description 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

[0001] This invention relates generally to computer 
testing, and more particularly to automated test summa- 
ry report generation. 

2. Description of the Related Art 

[0002] Business and other application programs exe- 
cute in a wide variety of operating environments. Before 
deploying an application program, it is often desirable 
to test the application program to determine its suitability 
for the operating environment and to discover any prob- 
lems associated with the application program before 
any undesirable physical, economic, or other damage 
may occur. 

[0003] As the number and complexity of application 
programs and operating environments continue to in- 
crease, along with the potential damage that defects 
and problems associated with application programs 
may cause in some circumstances, techniques for test- 
ing application programs have become increasingly im- 
portant. In conventional application program testing, a 
simulation of the operating environment in which the ap- 
plication program is intended to execute is generated. 
The application program is then executed within the sim- 
ulated operating environment and the test results ob- 
tained. 

[0004] Figure 1 is diagram showing a prior art testing 
cycle 100. During a conventional test cycle, a test appli- 
cation 102 is executed on a particular platform 104. 
Generally, the test application 1 02 includes a plurality of 
test suites, which are used to test various aspects of an 
environment. For example, the test application 102 can 
include a plurality of test suites to test a particular Java 
application program interface (API). 
[0005] The results of the test execution are then cap- 
tured in a test execution log file 106. The test execution 
log file 106 includes detailed descriptions of which tests 
were executed and the results of each test. Generally, 
the test execution log file 1 06 is a long document having 
very detailed information. As such, the test execution 
log file 106 can be difficult to interpret. Thus, the testing 
team running the tests often manually analyzes the test 
execution log file 1 06 to determine which tests pass and 
which tests fail. In the case of failures, the testing team 
uses the test execution log file to determine where the 
failures are occurring and why the failures are occurring. 
These results are then summarized in a test summary 
report 108. 

[0006] As the name implies, the test summary report 
108 provides a summary of the detailed testing informa- 
tion contained in the test execution log file 106. In addi- 
tion, the test summary report 108 includes a summary 



of the analysis performed on the test execution log file 
106. Often, the test summary file 108 lists the tests that 
were executed and whether the test passed or failed. 
For example, the test summary report of Figure 1 illus- 

5 trates two test suites, test suite X and test suite Y. Test 
suite X has thirty tests of which twenty-eight passed and 
two failed. Test suite Y has forty tests of which fourteen 
passed and twenty-six failed. Thus, to create the test 
summary 108 of Figure 1, the testing team must manu- 

10 ally analyze the test results in the test execution log file 
106 for seventy tests. 

[0007] The test summary report 1 08 is then distribut- 
ed to the appropriate personal, such as the project man- 
ager or development team. Although the conventional 

15 testing cycle 1 00 described above can provide test sum- 
mary reports 108, the conventional process of generat- 
ing the test summary report 108 is very time intensive 
and subject to errors. For example, manual counting of 
the tests pass and fail results is subject to counting er- 

20 rors, in which case the entire test suite would need to 
be reexamined. Moreover, conventionally over twenty 
hours per platform 104 is often needed to produce the 
test summary report 108. 

[0008] In view of the foregoing, there is a need for sys- 
25 terns and method for test report generation. The meth- 
ods should provide test summary reports in an automat- 
ed manner to reduce total test cycle time requirements 
and human error. 

30 SUMMARY OF THE INVENTION 

[0009] The embodiments of the present invention fill 
these needs by providing an Extensible Markup Lan- 
guage (XML) based report generator. The XML based 

35 report generator of the embodiments of the present in- 
vention allows a test summary report to be generated 
from a test execution log file quickly, generally without 
manual intervention from a user, and consequently, re- 
ducing human induced errors. In one embodiment, a 

40 method for creating a test summary report is disclosed. 
Broadly speaking, a test is executed and the test results 
are generated in a XML enabled format. The XML ena- 
bled test results are processed to create a test summary 
report. 

45 [0010] In another embodiment, a XML based report 
generator is disclosed. The XML based report generator 
includes a parser that processes a test execution log file 
to generate a well-formed XML test reports file. In addi- 
tion, a logical parser is included that processes the well- 

50 formed XML test reports file to produce a logically ar- 
ranged XML test reports file. The XML based report gen- 
erator further includes a Hypertext Markup Language 
(HTML) converter parser that converts the logically ar- 
ranged XML test reports file into a HTML test summary 

55 file. 

[0011] Another method for creating a test summary 
report is disclosed in a further embodiment of the 
present invention. The method includes executing a test 
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application on a platform, where the test application is 
executed using a status utility having functions that gen- 
erates XML code. The test results are generated in a 
XML enabled format using the status utility, and are out- 
put to a test execution log file. The test execution log file 5 
is processed to generate a well-formed XML test reports 
file, which is then arranged to create a logically arranged 
XML test reports file. The logically arranged XML test 
reports file is then converted into a HTML test summary 
report. Other aspects and advantages of the invention 
will become apparent from the following detailed de- 
scription, taken in conjunction with the accompanying 
drawings, illustrating by way of exampte the principles 
of the invention. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[001 2] The invention , together with further advantag- 
es thereof, may best be understood by reference to the 
following description taken in conjunction with the ac- 
companying drawings in which: 

Figure 1 is diagram showing a prior art testing cycle; 

Figure 2A is a block diagram showing a test execu- 
tion log file, in accordance with an embodiment of 
the present invention; 

Figure 2B is a block diagram showing a test suite 
listing, in accordance with an embodiment of the 
present invention; 

Figure 3 is a diagram showing an exemplary Test 
DTD, in accordance with an embodiment of the 
present invention; 

Figure 4 is a logical diagram showing a testing sys- 
tem, in accordance with an embodiment of the 
present invention; 

Figure 5 is a logical diagram showing a process cy- 
cle for generating a test summary report, in accord- 
ance with an embodiment of the present invention; 

Figure 6 is a flowchart showing a method for auto- 
mated XML based report generation, in accordance 
with an embodiment of the present invention; and 

Figure 7 is a block diagram of an exemplary com- 
puter system for carrying out the test processing ac- 
cording to the invention. 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

[0013] An invention is disclosed for a XML based re- 
port generator. The XML based report generator of the 
embodiments of the present invention allows a test sum- 
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many report to be generated from a test execution log 
file quickly, generally without manual intervention from 
a user, and consequently, reducing human induced er- 
rors. In the following description, numerous specific de- 
tails are set forth in order to provide a thorough under- 
standing of the present invention. It will be apparent, 
however, to one skilled in the art that the present inven- 
tion may be practiced without some or all of these spe- 
cific details. In other instances, well known process 
steps have not been described in detail in order not to 
unnecessarily obscure the present invention. 
[0014] Figure 1 was described in terms of the prior art. 
Figures 2A and 2B illustrate components of a test exe- 
cution log file. In particular, Figure 2A is a block diagram 
showing a test execution log file 200, in accordance with 
an embodiment of the present invention. As mentioned 
above, the test execution log file 200 is created during 
execution of the test application, which includes a plu- 
rality of test suites. For example, a test application can 
include a plurality of test suites to test a particular Java 
application program interface (API). 
[0015] The results of the test execution are captured 
in the test execution log file 200, which includes detailed 
descriptions of which tests were executed and the re- 
sults of each test. In particular, the test execution log file 
200 includes a platform listing 202 for each platform test- 
ed. Each platform listing 202 includes a plurality of test 
suite listings 204a-204b, each testing various aspects 
of the test environment. Additional test suite data 206 
can be listed after each test suite listing 204a-204b in- 
dicating various test information, such as debugging 
and logging data. The test suite data 206 can also be 
listed before the first test suite listing 204a. Although the 
test execution log file 200 is shown in Figure 2A with 
one platform listing 202 having two test suite listings 
204a and 204b, it should be noted that a test execution 
log file 200 of the embodiments of the present invention 
can include any number of platform listings 202, each 
having any number of test suite listings 204. 
[0016] Figure 2B is a block diagram showing a test 
suite listing 204a, in accordance with an embodiment of 
the present invention. The test suite listing 204a in- 
cludes a plurality of test listings 208a-208b, each listing 
the results of a particular test of the test suite 204a. Sim- 
ilar to test suite listings 204a, additional test data 216 
can be listed before the first test listing 208a and after 
each test listing 208a-208b indicating various test infor- 
mation, such as debugging and logging data. As above, 
although the test suite listing 204a is shown in Figure 
2B having two test listings 208a and 208b, it should be 
noted that a test suite listing 204a of the embodiments 
of the present invention can include any number of test 
listings 208. 

[0017] Each test listing 208a-208b lists compile test 
results 210a-210b, execute test results 212a-212b, and 
a test result 214a-214b. The compile test results 210a- 
210b list information on the test compilation for the par- 
ticular test 208a. For example, the compile test results 
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210a can list whether or not the test 208a compiled cor- 
rectly, and if not, source code errors. The execute test 
results 212a-212b list information on the test execution 
of the particular test 208a. For example, the execute test 
results 212a can list whether or not the test 208a exe- 5 
cuted correctly, and if not, the reason for the execution 
failure. The test results 214a-214b list the actual test 
output for the particular test. For example, test result 
214a can list the actual test output for test 208a, includ- 
ing whether the test passed or failed, and in the case of 
failure, why the failure occurred and where the failure 
occurred. To automate the test cycle, embodiments of 
the present invention define a XML document type def- 
inition (DTD) for the test result phase 214. 
[0018] XML is an open standard for describing data 
and is often used for defining data elements on a Web 
page and business-to-business documents. XML uses 
a similar tag structure as HTML. However, whereas 
HTML defines how elements are displayed, XML de- 
fines what those elements contain. Further, HTML uses 
predefined tags, while XML allows tags to be defined by 
the developer of the page. Thus, virtually any data items, 
such as test suites and individual tests, can be identified, 
allowing XML documents to function like database 
records. 

[0019] The human-readable XML tags provide a sim- 
ple data format, which is defined in a DTD format that 
defines content type as well as name. Thus, unlike 
HTML, which uses a rather loose coding style and which 
is tolerant of coding errors, XML pages are "well 
formed," which means they comply with rigid rules. 
[0020] An XML document primarily comprises a strict- 
ly nested hierarchy of elements with a single root. Ele- 
ments can contain character data, child elements, or a 
mixture of both. In addition, they can have attributes. 
Child character data and child elements are strictly or- 
dered, while attributes are not. The names of the ele- 
ments and attributes and their order in the hierarchy 
(among other things) form the XML markup language 
used by the document, also known as the "validity" of 
the document. As mentioned above, this language can 
be defined by the document author or it can be inferred 
from the document's structure. In particular, the embod- 
iments of the present invention define four elements: 
test, test name, test case, and test summary, as dis- 
cussed in greater detail subsequently with reference to 
Figure 3. 

[0021] Figure 3 is a diagram showing an exemplary 
Test DTD 300, in accordance with an embodiment of the 
present invention. The exemplary test DTD 300 defines 
the elements Test 302, Test-Name 304, Test-Case 306, 
and Test-Summary 308. The element Test 302 includes 
zero or one Test-Name element 304, zero or more 
Test-Case elements 306, and a Test-Summary element 
308. Test 302 has attributes Suite-Name 310 and Cat- 
egory 312. Suite-Name 310 identifies the test suite con- 
taining the test 302, and Category 312 can identify and 
API, product, or other classification the user desires. 
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Test-Case-ID provides an ID for each test case, and in- 
cludes a status attribute 314 that can have a value of 
passed or failed. Test-Case-Desc 316 provides a de- 
scription or reason for a failure. In addition, Test-Case- 
Desc 316 can provide a message for a passed test. 
Test-Cases-Passed 318 and Test-Cases-Failed 320 in- 
dicate the number of tests passing or failing respective- 
ly. 

[0022] The Test DTD 300 sets forth a format for use 
when reporting test results during application tests. In 
particular, defining the Test language as above allows 
an extensible stylesheet language transformation 
(XSLT) stylesheet parser to be used to process test re- 
sults formatted according to the Test DTD 300. Based 
on the Test DTD 300, embodiments of the present in- 
vention generate test results that conform the Test DTD 
300 via a status utility as described in greater detail next 
with reference to Figure 4. 

[0023] Figure 4 is a logical diagram showing a testing 
system 400, in accordance with an embodiment of the 
present invention. In operation, a test application 402 is 
executed on a platform 404. The test application 404 
includes a plurality of test suites, which are used to test 
various aspects of an environment, as mentioned above 
with respect to Figure 2A. For example, the test appli- 
cation 404 can include a plurality of test suites to test a 
particular Java application program interface (API). 
[0024] The results of the test execution are then cap- 
tured in a test execution log file 200 which includes de- 
tailed descriptions of the tests that were executed and 
the results of each test, as described above with respect 
to Figures 2A and 2B. To generate the test results in- 
cluded in the test execution log file 200, embodiments 
of the present invention make function calls to a status 
utility 406. The status utility 406 includes functions that 
generate XML statements in accordance with the test 
DTD 300, discussed above with respect to Figure 3. In 
particular, the test application 402 includes function calls 
to the functions provided in the status utility 406. These 
function return XML statements in accordance with the 
test DTD 300, which are then written to the test execu- 
tion log file 200. As a result, the test execution log file 
200 includes test results that are XML enabled, in addi- 
tion to non-XML enabled compiler and execution infor- 
mation as described above with reference to Figure 2B. 
[0025] Generally, the resulting test execution log file 
200 is a long document having very detailed information. 
As such, the test execution log file 200 can be difficult 
to interpret. Thus, embodiments of the present invention 
process the test execution log file 200 into a test sum- 
mary report using an automated process. Although the 
test execution log file 200 is process to create a test 
summary report, it will be noted that the test execution 
log file 200 can be saved and referred to when needed 
at a later date. For example, since the test execution log 
file 200 often includes compiling and execution informa- 
tion, this information can be later examined to determine 
the precise environment used for the test. 
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[0026] Embodiments of the present invention can also 
include the functions of the status utility 406 within the 
test application 402 document itself, or as part of a lan- 
guage library. In such embodiments, a separate status 
utility 406 may not be needed, since the functions will 5 
be present in other code elements, for example, within 
the test application 402 itself. In further embodiments, 
print statements or character output statements can be 
used to generate the XML enabled test results. In such 
embodiments, the print statements should create state- 
ments that are XML enabled according to a particular 
Test DTD, such as the exemplary Test DTD 300 of Fig- 
ure 3, as defined by the test developer. Regardless of 
the embodiment used to create the test execution log 
file 200, the test execution log file 200 is subsequently 
processed to create a well-formed XML document, 
which is used to create a test summary report. 
[0027] Figure 5 is a logical diagram showing a proc- 
ess cycle 500 for generating a test summary report, in 
accordance with an embodiment of the present inven- 
tion. As shown in Figure 5, the test execution log file 200 
is input to a parser 502, which processes the test exe- 
cution log file 200 to produce a well-formed XML test 
report file 504. It should be noted that the parser 502 
can also be implemented as a Java utility using a Java 
code. As mentioned above, the test execution log file 
200 includes information other than test result informa- 
tion. For example, the test execution log file 200 often 
includes compiling and execution information that is 
used in debugging and other test maintenance opera- 
tions. The parser 502 processes this information extra 
information to create the well-formed XML test report file 
504. In addition, the parser 502 can extract control char- 
acters not utilized during further operations of the proc- 
ess. 

[0028] In one embodiment, the parser 502 detects 
compiler and execution information and other test infor- 
mation not enclosed in XML tags and relocates the data 
within CDATA tags. The CDATA tags can then be further 
processed or ignored by subsequent parsers. General- 
ly, the data within the CDATA tags includes information 
not relevant to the results and thus is generally ignored 
by the parser. In further embodiments, compiler and ex- 
ecution information and other test information not en- 
closed in XML tags within the test execution log file 200 
can be discarded. 

[0029] As mentioned above, the parser 502 creates a 
well-formed XML test report file 504. In addition, embod- 
iments of the present invention create the well-formed 
XML test report file 504 such that the XML enabled test 
reports are valid as well, according to the Test DTD. A 
well-formed XML document is a XML document that 
complies with XML well-formedness constraints. These 
constraints require that elements, which are named con- 
tent containers, properly nest within each other and use 
other markup syntax correctly. Unlike HTML, well- 
formed XML elements are defined by their use, not by 
a rigid structural definition, allowing authors to create 



elements in response to their development. A valid XML 
document is a XML document that conforms with a cor- 
responding DTD. As mentioned above, A DTD is a set 
of rules that a document follows, which software may 
need to read before processing and displaying a docu- 
ment. These rules generally state the name and con- 
tents of each element and in which contexts it can exist. 
Paragraph elements might be defined as containing 
keyword and code elements and as existing within sec- 
tion and note elements. 

[0030] Although the well-formed XML test report file 
504 is well-formed and valid according to the Test DTD, 
it is generally desirable to logically arrange the well- 
formed XML test report file 504 based on the test suites. 
Thus, the well-formed XML test report file 504 is proc- 
essed by a logical XSLT stylesheet parser 506, which 
rearranges the well-formed XML test report file 504 into 
a logically arranged XML test report file 508. 
[0031] As each test is executed, the results are written 
to the test execution log file 200 using the status utility, 
as described above with reference to Figure 4. However, 
during a particular test cycle, a test engineer may run 
any number of tests that are available in a particular test 
suite. That is, although a particular test suite may in- 
clude a predefined number of tests, the test engineer 
may run all, some, or none of the tests within the partic- 
ular test suite. Thus, the actual number of test executed 
in a particular test suite may not be known until the test 
suite is actually executed. The embodiments of the 
present invention address this issue by writing test re- 
sults to the test execution log file 200 in an independent 
manner. Specifically, each test result includes informa- 
tion identifying the test and the test suite to which the 
test belongs. Hence, the test results of the well-formed 
XML test reports file 504 are arranged as a plurality of 
independent test results, each identifying its test ID and 
the test suite to which the test belongs. For example, 
using the exemplary DTD 300 of Figure 3, the 
Suite-Name attribute 310 can be used to logically ar- 
range the tests. In this embodiment, all test belonging 
to the same test suite, as indicated by the Suite-Name 
attribute 310, are arranged together under one XML el- 
ement. 

[0032] In one embodiment, the logical XSLT style- 
sheet parser 506 is written using XSLT. XSLT is a lan- 
guage used to convert a XML document into another 
XML document or into HTML, PDF, or some other for- 
mat. The conversion is accomplished with a XSLT proc- 
essor, which transforms the input based on XSLT exten- 
sions of the XSL stylesheet. XSL statements are also 
followed. The processor uses a XML parser to separate 
the XML elements into a tree structure, which the proc- 
essor manipulates. Although, a logical XSLT stylesheet 
parser 506 is illustrated in Figure 5, it should be noted 
that the logical parser 506 of the embodiments of 
present invention can be developed utilizing any com- 
puter programming language, such as Java, C, Assem- 
bly, or other computer programming languages as will 
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be apparent to those skilled in the art. 
[0033] The logical XSLT stylesheet parser 506 proc- 
esses the well-formed XML test report file 504 into a log- 
ically arranged XML test report file 508. Specifically, the 
logical XSLT stylesheet parser 506 analyses the test re- 5 
suit data of the well-formed XML test report file 504 to 
determine which test suites include which test results. 
The logical XSLT stylesheet parser 506 generates tags 
for each test suite so determined and encapsulates the 
test results corresponding to each test suite within the 
tags of the test suite. The new logically arranged test 
results are then written to the logically arranged XML 
test report file 508. 

[0034] Once the test results are logically arranged 
within the logically arranged XML test report file 508, a 
HTML converter XSLT stylesheet parser 510 converts 
the logically arranged XML test report file 508 into a 
HTML test summary report 512. As the name implies, 
the HTML test summary report 512 comprises HTML 
code, which can be interpreted using a browser. HTML 
is the set of markup symbols or codes inserted in a file 
intended for display on a browser. The markup symbols 
are a sequence of characters or other symbols that are 
inserted at certain places in a text or word processing 
file to indicate how the file should look when it is printed 
or displayed or to describe the document's logical struc- 
ture. These markup indicators are often called "tags." 
[0035] The markup tells the browser how to display a 
HTML page's words and images for the user. Each in- 
dividual markup code is referred to as a "tag." Some tags 
come in pairs that indicate when a particular display ef- 
fect is to begin and when it is to end. HTML is generally 
adhered to by the major browsers, Microsoft's Internet 
Explorer and Netscape's Navigator, which also provide 
some additional non-standard codes. 
[0036] For example, the HTML test summary report 
51 2 can be displayed on a browser 51 4, thus, presenting 
the user with a summary of the detailed testing informa- 
tion included in the test execution log file 200. In addi- 
tion, the HTML test summary report 51 2 includes a sum- 
mary analysis of the test execution log file 200. Specif- 
ically, the HTML test summary report 512 can list the 
tests that were executed and whether the tests passed 
or failed. For example, the browser displayed HTML test 
summary report 514 of Figure 5 illustrates two test 
suites, test suite X and test suite Y. Test suit X has thirty 
tests of which twenty-eight passed and two failed. Test 
suit Y has forty tests of which fourteen passed and twen- 
ty-six failed. 

[0037] In addition, since the HTML test summary re- 
port 512 is written in HTML, a link 515 can be provided 
for test failures. The link 515 provides access to other 
HTML pages that describe the failure and why the failure 
occurred. In the case of failures, embodiments of the 
present invention can use the test execution log file 200 
to determine where the failures are occurring and why 
the failures are occurring. These results can then be 
summarized in the HTML test summary report 512 and 



accompanying failure description pages 516. For exam- 
ple, when a user selects a link 51 5 for the failures of test 
suite X, the user is presented with a failure description 
page 516 describing the test failures of test suit X and 
why the failures occurred. The HTML test summary re- 
port 512 can then distributed to the appropriate person- 
al, such as the project manager or development team. 
Embodiments of the present invention can also describe 
test failures within the same document as the test sum- 
mary report 512 and use local links to access the failure 
descriptions. 

[0038] Figure 6 is a flowchart showing a method 600 
for automated XML based report generation, in accord- 
ance with an embodiment of the present invention. In 
an initial operation 602, preprocess operations are per- 
formed. Preprocess operations include generating test 
suites, determining which tests within the test suites to 
execute, and other preprocess operation that will be ap- 
parent to those skilled in the art. 

[0039] In operation 604, the test application is execut- 
ed and a test execution log file is generated. Generally, 
the test application includes a plurality of test suites, 
which are used to test various aspects of an environ- 
ment. The results of the test execution are captured in 
a test execution log file, which includes detailed descrip- 
tions of the tests that were executed and the results of 
each test. 

[0040] To generate the test results included in the test 
execution log file, embodiments of the present invention 
make function calls to a status utility, which includes 
functions that generate XML statements in accordance 
with a predefined test DTD. In particular, the test appli- 
cation includes function calls to the functions provided 
in the status utility, which return XML statements in ac- 
cordance with the test DTD. The returned XML state- 
ments are then written to the test execution log file. As 
a result, the test execution log file includes test results 
that are XML enabled, in addition to non-XML enabled 
compiler and execution information. 
[0041 ] In operation 606, a well-formed XML test report 
file is created using the test execution log file. The test 
execution log file is input to a parser, which processes 
the test execution log file to produce a well-formed XML 
test report file. As mentioned above, the test execution 
log file includes information other than test result infor- 
mation. For example, the test execution log file often in- 
cludes compiling and execution information that is used 
in debugging and other test maintenance operations. 
The parser processes this extra information to create 
the well-formed XML test report file. In addition, as de- 
scribed above, the well-formed XML test report file gen- 
erally is valid with respect to the test DTD. 
[0042] In one embodiment, the parser detects compil- 
er and execution information and other test information 
not enclosed in XML tags and relocates the data within 
CD ATA tags. The CDATA tags can then be further proc- 
essed or ignored by subsequent parsers. 
[0043] The test results are logically arranged per the 
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test suites, in operation 608. As each test is executed, 
the results are written to the test execution log file using 
the status utility in an independent manner. That is, each 
test result includes information identifying the test and 
the test suite to which the test belongs. Hence, the test 
results of the well-formed XML test report file are ar- 
ranged as a plurality of independent test results, each 
identifying its test ID and the test suite to which the test 
belongs. A logical XSLT stylesheet parser processes 
the well-formed XML test report file into a logically ar- 
ranged XML test report file. Specifically, the logical 
XSLT stylesheet parser analyses the test result data of 
the well-formed XML test report file to determine which 
test suites include which test results. The logical XSLT 
stylesheet parser generates tags for each test suite and 
encapsulates the test results corresponding to each test 
suite within the tags of the test suite. The new logically 
arranged test results are then written to the logically ar- 
ranged XML test report file. 

[0044] Once the test results are logically arranged 
within the logically arranged XML test report file, a HTML 
converter XSLT stylesheet parser converts the logically 
arranged XML test report file into a HTML test summary 
report, in operation 610. The HTML test summary report 
comprises HTML code, which can be interpreted using 
a browser. The HTML test summary report can be dis- 
played on a browser, thus, presenting the user with a 
summary of the detailed testing information included in 
the test execution log file. In addition, the HTML test 
summary report can include a summary analysis of the 
test execution log file. Specifically, the HTML test sum- 
mary report can list the tests that were executed and 
whether the tests passed or failed. 
[0045] In addition, since the HTML test summary re- 
port is written in HTML, a link can be provided for test 
failures. The link provides access to other HTML pages 
that describe the failure and why the failure occurred. In 
the case of failures, embodiments of the present inven- 
tion can use the test execution log file to determine 
where the failures are occurring and why the failures are 
occurring. These results can then be summarized in the 
HTML test summary report and accompanying failure 
description pages. Post process operations are per- 
formed in operation 612. Post process operations can 
include distributing the HTML test summary report to ap- 
propriate personal, such as the project manager or de- 
velopment team, and other post process operations that 
will be apparent to those skilled in the art. 
[0046] Embodiments of the present invention may 
employ various computer-implemented operations in- 
volving data stored in computer systems to drive com- 
puter software, including application programs, operat- 
ing system programs, peripheral device drivers, etc. 
These operations are those requiring physical manipu- 
lation of physical quantities. Usually, though not neces- 
sarily, these quantities take the form of electrical or mag- 
netic signals capable of being stored, transferred, com- 
bined, compared, and otherwise manipulated. Further, 



the manipulations performed are often referred to in 
terms, such as producing, identifying, determining, or 
comparing. 

[0047] Any of the operations described herein that 
5 form part of the invention are useful machine operations. 
The invention also relates to a device or an apparatus 
for performing these operations. The apparatus may be 
specially constructed for the required purposes, or it 
may be a general purpose computer selectively activat- 
ed or configured by a computer program stored in the 
computer. In particular, various general purpose ma- 
chines may be used with computer programs written in 
accordance with the teachings herein, or it may be more 
convenient to construct a more specialized apparatus 
to perform the required operations. An exemplary struc- 
ture for the invention is described below. 
[0048] Figure 7 is a block diagram of an exemplary 
computer system 700 for carrying out the processing ac- 
cording to the invention. The computer system 700 in- 
cludes a digital computer 702, a display screen (or mon- 
itor) 704, a printer 706, a floppy disk drive 708, a hard 
disk drive 710, a network interface 712, and a keyboard 
714. The digital computer 702 includes a microproces- 
sor 716, a memory bus 718, random access memory 
(RAM) 720, read only memory (ROM) 722, a peripheral 
bus 724, and a keyboard controller (KBC) 726. The dig- 
ital computer 702 can be a personal computer (such as 
an IBM compatible personal computer, a Macintosh 
computer or Macintosh compatible computer), a work- 
station computer (such as a Sun Microsystems or 
Hewlett-Packard workstation), or some other type of 
computer. 

[0049] The microprocessor 71 6 is a general purpose 
digital processor which controls the operation of the 
computer system 700. The microprocessor 716 can be 
a single-chip processor or can be implemented with mul- 
tiple components. Using instructions retrieved from 
memory, the microprocessor 716 controls the reception 
and manipulation of input data and the output and dis- 
play of data on output devices. According to the inven- 
tion, a particular function of microprocessor 716 is to ex- 
ecute the test application to generate the test execution 
log. In addition, the microprocessor 71 6 further controls 
the execution of the parser, Logical XSLT stylesheet 
parser, and HTML converter XSLT stylesheet parser. 
[0050] The memory bus 71 8 is used by the microproc- 
essor 716 to access the RAM 720 and the ROM 722. 
The RAM 720 is used by the microprocessor 716 as a 
general storage area and as scratch-pad memory, and 
can also be used to store input data and processed data. 
The ROM 722 can be used to store instructions or pro- 
gram code followed by the microprocessor 716 as well 
as other data. 

[0051] The peripheral bus 724 is used to access the 
input, output, and storage devices used by the digital 
computer 702. In the described embodiment, these de- 
vices include the display screen 704, the printer device 
706, the floppy disk drive 708, the hard disk drive 710, 
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and the network interface 712. The keyboard controller 
726 is used to receive input from keyboard 714 and send 
decoded symbols for each pressed key to microproces- 
sor 716 over bus 728. 

[0052] The display screen 704 is an output device that 5 
displays images of data provided by the microprocessor 
71 6 via the peripheral bus 724 or provided by other com- 
ponents in the computer system 700. The printer device 
706, when operating as a printer, provides an image on 
a sheet of paper or a similar surface. Other output de- 
vices such as a plotter, typesetter, etc. can be used in 
place of, or in addition to, the printer device 706. 
[0053] The floppy disk drive 708 and the hard disk 
drive 710 can be used to store various types of data. 
The floppy disk drive 708 facilitates transporting such 
data to other computer systems, and hard disk drive 710 
permits fast access to large amounts of stored data. 
[0054] The microprocessor 716 together with an op- 
erating system operate to execute computer code and 
produce and use data. The computer code and data 
may reside on the RAM 720, the ROM 722, or the hard 
disk drive 710. The computer code and data could also 
reside on a removable program medium and loaded or 
installed onto the computer system 700 when needed. 
Removable program media include, for example, 
CD-ROM, PC-CARD, floppy disk and magnetic tape. 
[0055] The network interface 71 2 is used to send and 
receive data over a network connected to other compu- 
ter systems. An interface card or similar device and ap- 
propriate software implemented by the microprocessor 
716 can be used to connect the computer system 700 
to an existing network and transfer data according to 
standard protocols. 

[0056] The keyboard 714 is used by a user to input 
commands and other instructions to the computer sys- 
tem 700. Other types of user input devices can also be 
used in conjunction with the present invention. For ex- 
ample, pointing devices such as a computer mouse, a 
track ball, a stylus, or a tablet can be used to manipulate 
a pointer on a screen of a general-purpose computer. 
[0057] The invention can also be embodied as com- 
puter readable code on a computer readable medium. 
The computer readable medium is any data storage de- 
vice that can store data which can be thereafter be read 
by a computer system. Examples of the computer read- 
able medium include read-only memory (ROM), ran- 
dom-access memory (RAM), CD-ROMs, magnetic tape, 
and optical data storage devices. The computer reada- 
ble medium can also be distributed over a network that 
couples computer systems so that the computer reada- 
ble code is stored and executed in a distributed fashion. 
[0058] Furthermore, the same or similar methods and 
apparatuses described above for programming a hard- 
ware device can also be used for performing other par- 
ticular maintenance operations on the hardware device. 
For example, operations such as erasing a ROM, read- 
ing a ROM, or performing a checksum on a ROM can 
be performed. 



[0059] Although the foregoing invention has been de- 
scribed in some detail for purposes of clarity of under- 
standing, it will be apparent that certain changes and 
modifications may be practiced within the scope of the 
appended claims. Accordingly, the present embodi- 
ments are to be considered as illustrative and not re- 
strictive, and the invention is not to be limited to the de- 
tails given herein, but may be modified within the scope 
and equivalents of the appended claims. 
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Claims 

1. A method for creating a test summary report, com- 
15 prising the operations of: 

executing a test; 

generating test results (200) in an Extensible 
Markup Language (XML) enabled format; and 
20 processing the XML enabled test results to cre- 

ate a test summary report (512). 

2. A method as recited in claim 1 , wherein the test re- 
sults are generated utilising a status utility having 

25 functions that generate XML code. 

3. A method as recited in claim 1 or 2 wherein the test 
results are output to a test execution log file, the test 
execution log file including a log of the test execu- 

30 tion. 

4. A method as recited in claim 3, further including the 
operation of processing the test execution log file 
(200) to generate a well-formed XML test reports 

35 file (504). 

5. A method as recited in claim 4, wherein the well- 
formed XML test reports file (504) is further valid 
with respect to a Test document type definition 

40 (DTD). 

6. A method as recited in claim 4 or 5 further compris- 
ing the operation of logically arranging the well- 
formed XML test reports file (504) to create a logi- 
cs cally arranged XML test reports file (508). 

7. A method as recited in claim 6, wherein the logically 
arranged XML test reports file (508) includes test 
suit tags indicating test reports that belong to par- 

50 ticular test suites. 

8. A method as recited in claim 6 or 7 wherein the test 
summary report is generated by converting the log- 
ically arranged XML test reports file (508) into a 

55 HTML test summary report (512). 

9. A method as recited in claim 8, wherein the HTML 
test summary report (512) provides a test summary 
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of the test execution log file (200). 

10. A method as recited in claim 9, wherein the HTML 
test summary report (512) includes links to failure 
description pages, wherein the failure description 5 
pages provide a detailed description of a particular 
test failure. 

11. An Extensible Markup Language (XML) based re- 
port generator, comprising: 10 

a parser (502) that processes a test execution 
log file (200) to generate a well-formed XML 
test reports file (504); 

a logical parser (506) that processes the well- 15 
formed XML test reports file (504) to produce a 
logically arranged XML test reports file (508); 
and 

a HTML converter parser (510) that converts 
the logically arranged XML test reports file 20 
(508) into a HTML test summary file (512). 

12. A XML based report generator as recited in claim 
11, wherein the well-formed XML test reports file 
(504) is valid with respect to a Test document type 25 
definition (DTD). 

13. A XML based report generator as recited in claim 
11 or 12, wherein the logically arranged XML test 
reports file (508) includes test suite tags indicating 30 
test reports that belong to particular test suites. 

14. A XML based report generator as recited in claim 
1 1 , 1 2 or 1 3 wherein the HTML test summary report 

(51 2) provides a test summary of the test execution 35 
log file. 

15. A XML based report generator as recited in any of 
claims 11 to 14 wherein the HTML test summary re- 
port (512) includes links to failure description pag- 40 
es, wherein the failure description pages provide a 
detailed description of a particular test failure. 

16. A storage medium storing computer implementable 
computer instructions for causing a programmable 45 
computer to implement a method in accordance 
with any of claims 1 to 10 or to cause a program- 
mable computer to become configured as an exten- 
sible mark-up language based report generator in 
accordance with any of claims 11 to 15. 50 
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